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(57) ABSTRACT 

Methods, systems, devices and the lilce for measuring, 
encoding and displaying of object color for digital imaging, 
to control the apparent color of an item under different 
lighting or display conditions. The present invention helps 
provide accurate color for such items, both on the internet 
and in other situations, and can, if desired, determine 
whether a display screen is accurately di^laying the color 
and also correct the image on the screen if it is not accurate. 
Also, probes suitable for taking accurate measurements of 
the intrinsic color characteristics, or intrinsic wavelength- 
dependent response, of an object, and software or databases 
that provide information for a variety of lighting situations 
and light sources. 
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APPARATUS AND METHOD FOR 
MEASUREMENT, ENCODING AND DISPLAYING 
OF OBJECT COLOR FOR DIGITAL IMAGING 

CROSS-REFERENCE TO RELATED 
APPLICATION 

[0001] The present applicatioa claims priority from U.S. 
provisional patent application No. 60/204,962, filed May 17, 
2000. 

FIELD 

[0002] This invention relates to measurement and charac- 
terization of object interaction with visible light by spectro- 
photometry, calculating the expected reflectance under vari- 
o;is conditions of illumination, and encoding this 
information to display an accurate color rendering of the 
object when a digital image of the object is transmitted to a 
display device for viewing. 

BACKGROUND 

[0003] The apparent color of an item, such as a shirt or a 
car, can change dramatically depending on the light shining 
on the item. For example, a dress can look great under the 
soft lighting at the store then look ghastly under the harsh 
fluorescent lights at work. Similarly, people shoppmg on the 
internet are often leery of buying something that looks good 
on their computer screen because they cannot be sure how 
it will look in real life. The present invention provides 
systems and methods that help provide accurate color for 
such items, both on the internet and in other situations 
including, if desired, determining whether the computer 
screen is tuned to accurately show the color. Even further, 
the present invention provides systems and methods can that 
help show the item under different lighting conditions, such 
as outside in Seattle on a cloudy day or in Cairo on a sunny 
day, or even under those fluorescent lights at work. These 
concepts are discussed in more scientific terms in the 
following paragraphs. 

[0004] Discussion of the measurement of hght uses a 
variety of terms and measurement units. Unless otherwise 
indicated in the text, for example by express statement or by 
the context, terms relating to measurement and character- 
ization of light and color are defined by reference to Color 
Appearance Models, Mark D. Fairchild, Focal Press, Addi- 
son Wesley Longman, 1998 and the Handbook of Optics, 
CD-ROM Second Edition, sponsored by the Optical Society 
of America and published by McGraw-Hill, 1996. 

[0005] The advent of internet commerce or e-business has 
resulted in a demand for on-line catalogues of products for 
consimaer and business use. The most active and growth- 
oriented businesses tend to be technology related. Many 
customers employed at these businesses use the internet to 
source products required in their business. Businesses pro- 
ducing catalogues and brochures now require not just pho- 
tographs and drawings of their products, but digital images 
suitable for on-line catalogues and brochures. This is one 
factor, as weU as other factors such as digital pre-press and 
press technology in the printing industry, pushing product 
photography towards digital imaging. However, photogra- 
phers that are typically experienced with the color charac- 
teristics of film -based systems may not be experienced with 
digital systems. This can often incur extra uncertainty, costs 



and delays correcting color in digital images. Additionally, 
while the product vendor often has control over how the 
image is captured, there can be a number of problems in the 
image display process that can interfere with product pre- 
sentation. 

[0006] Turning to a general discussion of the namre of 
light, as well as color, color perception and color imaging, 
Hght is a form of energy. It has been described as both 
electromagnetic waves and photons. The color of light is 
related to the amount of energy carried by the photon or 
electromagnetic wave. The color of light is most commonly 
referred to by its wavelength. Wavelength is typically mea- 
sured in micrometers (10~* m) or nanometers (10^^ m). The 
relative distribution of photons of various wavelengths (or 
energies) of a given source of light is known as the energy 
spectrum (commonly abbreviated to spectrum) of the light 
source. 

[0007] Examples of light sources include the sun, regular 
incandescent lights, and fluorescent fights. The solar illumi- 
nation at the Earth's surface typically varies a few percent 
across a 20 nm step in the visible wavelength range and 
intensity of an incandescent filament lamp typically varies 
no more than 10% across a 20 mn increment of wavelength 
in this range. Conversely, a fluorescent lamp can have a 20 
times (2000%) change in intensity across a step of less than 
10 nm. Such variations in energy spectra can present diflS- 
culties when trying to provide accurate color rendition of an 
object. 

[0008] Similar but less dramatic spikes in illumination 
profile occur with xenon flash lamps used in photography, 
and other arc lamp sources. When a lamp with irregular 
spectral emission illuminates an object with irregular spec- 
tral reflection, color appearance can change significantly. 
For example, as noted above, a dress that looked good under 
halogen lamps at the store can look terrible in the fluorescent 
fights of an oflBce. The phenomenon of an object that 
changes in perceived color under different types of nomi- 
nally "white light" illumination is known as metamerism. A 
related effect is the commonly observed greenish cast in the 
color of a photograph caused by the green spike in fluores- 
cent fights when the film color response is balanced for 
dayfight or tungsten lighting. 

[0009] The light from the light source shines on a given 
object; the relative amoimt of light reflected by an object 
under iUumination is called its reflectance spectrum. The 
reflectance spectrum is one part of the set of intrinsic color 
characteristics, which can be referred to as the "intrinsic 
wavelength-dependent response," of any given object, such 
as a car on the street or an apple on a table. Typically, the 
apparent color of an object to a human eye is a result of the 
reflectance spectrum of the object under a given energy 
spectrum. Other factors such as fluorescence can also affect 
the apparent color of the object. The apparent color of a 
given object under a given iUumination can be referred to as 
the illumination-dependent color of the object. These and 
other color diaracteristics can be measured. 

[0010] The human eye responds differently to different 
wavelengths. It is more sensitive to some wavelengths than 
others. The typical hiunan eye can detect wavelengths of 
fight from about 400 nm to about 700 nm. Human color 
vision is trichromatic; the eye's color detectors, known as 
cone cells, detect three overlapping ranges of wavelengths. 
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generally in the red, green and blue ranges. The brain 
determines the relative response of the three kinds of color 
photoreceptors of the eye and interprets this as color. 

[0011] As technology has evolved people have measured 
light and color using various instnunents. Color rendition in 
photographic or digital imaging is the process of trying to 
encode enough of the intrinsic color characteristics of an 
object being illuminated and imaged so that the human eye, 
when viewing the object in the rendered image, can interpret 
the color as intended by the image creator. In order to render 
color appropriately, the image creator preferably has 
adequate knowledge of the color characteristics of the light 
source or illumination, the object being imaged, and the 
device and recording medium so that the image creator can 
adjust the image as desired. 

[0012] These and other concepts relating to light and color 
characteristics are discussed, for example, in the following 
references. U.S. Pat. Nos. 5,555,085; 5,369,481; 4,867,563; 
Handbook of Optics, CD-ROM Second Edition, sponsored 
by the Optical Society of America, McGraw-Hill, 1996; 
Professional Lighting Handbook, Second Edition" V. Carl- 
son, S. E. Carlson, Focal E*ress, Butterworth-Heinemann, 
1991; Color i^jpearance Models, Mark D. Fairchild, Focal 
Press, Addison Wesley Longman, 1998; Electronic Display 
Measurement, Peter A. Kellar, John Wiley & Sons, Inc., 
1997; Measuring Colour, 3"** Edition, R. W. G. Hunt, Foun- 
tain press, England, 1998; Color Technology for Electronic 
Imaging Devices, Henry R. Kang, SPIE Press, Bellingham, 
Wash. 1997; Understanding Color, Giordano Beretta, 
Hewlett-Packard Company, Palo Alto. 2000, 

[0013] The field of digital imaging has expanded greatly in 
recent years, particularly in response to the growth of the 
internet. Products that were previously marketed using 
printed brochures, catalogs, and in magazine advertise- 
ments, are now marketed over the internet via web sites and 
e-mail. In print media, on the internet and other electronic 
media (and elsewhere), marketers spend a great deal of time 
ensuring that the photographic images portraying their prod- 
ucts are accurately or appealingly reproduced. Color is a 
particularly important aspect of product marketing. When 
such images are digitized and/or compressed for transmis- 
sion over the internet a great deal of this control disappears 
and color presentation of products becomes undesirably 
variable. Additionally, although digital imaging is subject to 
a nurnber of variables that can degrade image presentation 
and aocwacy, digital imaging provides opportunities to 
control and correct images. There has gone unmet a need to 
sufficiently define and control how an object's color is 
presented in a digital image, on the internet and elsewhere. 

[0014] The present invention can, for example, capture 
and/or encode accurate intrinsic color characteristics, or 
intrinsic wavelength-dependent response, of an object. The 
system associates the color information with an image of the 
object that can be interpreted by hardware or software such 
as a plug-in component of an internet browser or other 
program. The information can be used to condition the 
image hue for more accurate or desirable color rendition by 
adjusting color to resemble a variety of possible illiunination 
conditions. Additionally, the software can provide compen- 
sation for the characteristics of the image display hardware 
and hardware settings of the user's system, as well as 



provide tools to adjust the display system for optimum color 
rendition. The present invention provides these and other 
related advantages. 

SUMMARY 

[0015] The present invention comprises apparatus, sys- 
tems, algorithms^ and computer-implemented methods and 
programming, etc., that provide accurate color measurement 
of an object with adequate resolution to effectively define 
how its appearance would be affected by differing types and 
qualities of illumination. The present invention also com- 
prises methods to digitally encode such information as data 
within or associated with a digital image. Algorithms can 
interrogate the display device showing the image and correct 
the color representation of an object in the image to com- 
pensate for various display devices that may be used. This 
invention further comprises algorithms, datasets or data- 
bases that define iUuminadon or energy spectra based on 
factors such as global position, date, time of day, and 
environmental conditions, as well as methods and systems 
that can modify the color representation of the object to 
accurately correspond to such illumination conditions. The 
image of the object can be static, such as a photograph, or 
it can comprise movement, which typically means the object 
is depicted in a sequence of images or depictions that 
together indicate movement or diange, such as a movie or 
television show. In addition, the object can be real or virtual. 

[0016] The invention further comprises a system of com- 
puter-controlled measurement devices, measurement 
probes, as well as software for calibrating the measurement 
devices and probes, for collecting and controlling measure- 
ments, analyzing measurements, and storing the information 
and results related to such analyses in a computer-readable 
format. The invention also provides software for associating 
the measiued and analyzed values of an object with a digital 
image of that object, and modifying the digital image so that 
the color of the object can be represented accurately, or 
otherwise as desired, on a variety of display devices and 
under a variety of proposed illumination conditions. 

[0017] In addition, the adjustment of the image according 
to the present invention can be performed in a single 
computer that both displays the image and contains the 
programming that controls the adjustments, or the adjust- 
ment can be performed remotely, wherein one computer, 
such as a central computer, contains the adjustment pro- 
gramming and another, remote terminal or computer com- 
prising such a remote terminal displays the image. Examples 
of such computer networks include local area networks, 
fiber optic networks, the internet and the World Wide Web; 
the networks can, for example, be hardwired or operably 
connected by wireless apparatus such as cell phones, fi^- 
spaoe optical networks, satellite networks or palm devices. 

[0018] In one aspect the present invention provides a 
computer-implemented methods of adjusting at least one 
color of at least one object in a digital image comprising: a) 
providing the image containing the object and determining 
the chrominance values of the object in the image; b) 
providing a reference data set comprising an illumination- 
dependent color of the object that represents the color of the 
object under a desired illumination condition; and, c) adjust- 
ing the chrominance values of the object in the image using 
the reference data set to provide a different, desired color of 
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the object in the image that represents the color of the object 
under the desired illumination condition. 

[0019] In certain embodiments, the image includes at least 
a second object in the image, and the methods further 
comprise segmenting the object from a remainder of the 
image. The image can comprise pixels and the method can 
fiu-ther comprise adjusting all pixels in the image that have 
chrominance or luminance values similar to the chromi- 
nance or luminance values of the object substantially the 
same as pixels corresponding to the object. 

[0020] In additional embodiments, the methods can fur- 
ther or separately comprise obtaining the illumination-de- 
pendent color by: a) providing an intrinsic-object color of 
the object to iUumioation; b) providing at least one illumi- 
nation light source reference data set comprising wave- 
length-dependent intensity characteristics of a desired illu- 
mination light source sufficient to define the wavelength- 
dependent energy distribution of the desired illumination 
light source; and, c) combining a tristimulus function and the 
intrinsic object-color of the object and the illumination light 
source reference data set to calculate at least one set of 
tristimulus values for the object when illuminated by the 
desired illumination light source, and therefrom determining 
the illumination-dependent color of the object. 

[0021] The methods can also comprise adjusting the image 
at a remote terminal of a computer network with the com- 
puter-implemented programming able to perform the 
method installed on the remote terminal in combination with 
the image, or with such programming operating from a 
different location such as a central location. The computer 
network can comprise, for example, an internet network, a 
fiber optic network, or a cell phone as a carrier of the 
computer signal. The adjusting can adjust for any one or 
more of different artificial lights, different latitudes, different 
altitudes, different times of day and different weather situ- 
ations. The illumination-dependent color of the object can be 
an artificially created illumination-dependent color, and the 
intrinsic-object color of the object can be obtained using a 
numerical or graphical representation of the iUumination- 
dependent color of the object in combination with a spec- 
trum of a reference light source illuminating the image. The 
methods can alternatively comprise adjusting the iUumina- 
tion-dependent color of the object using gamut-mapping 
wherein the gamut can be defined by image rendering 
capabilities of an image rendering device. 

[0022] The image can be associated with an identifiable 
tag that invokes the method of adjusting or with an identi- 
fiable tag that permits the method of adjusting to be turned 
on by an external actuator. The identifiable tag can be a file, 
and the adjusting can be associated with a switch operably 
linked to be turned on by the identifiable tag. The methods 
can be performed by a computer program that can be 
combined with the image in a single file. The object can 
comprise a depiction of a real item and the intrinsic object- 
color of the real item can be an intrinsic object-color of the 
real item. The object can also comprise a depiction of an 
artificially-created object and the intrinsic object-color of the 
artificially-created object comprises predetermined assigned 
chrominance values. 

[0023] The methods typically further comprise determin- 
ing the luminance values of the object in the image and if 
desired adjusting the luminance values or using the lumi- 



nance values to determine pixels to be adjusted. The meth- 
ods can also comprise limiting the colors to be corrected to 
chrominance values characteristic of diffuse reflection. 

[0024] In other aspects the present invention provides a 
computer comprising computer-implemented programming 
able to perform the methods described herein, as well as a 
computer-readable memory comprising such computer- 
implemented programming. The present invention also pro- 
vides systems comprising probes able to detect an intrinsic 
object-color of an object to illumination, typically with a 
computer comprising computer-implemented programming 
as described herein. Preferably, the probe resolves the intrin- 
sic object-color to about 5 nm, further preferably to about 2 
nm. The probe can be a spectroradiometer. The systems can 
comprise an illumination light source having a known 
energy spectmm, preferably the system comprises com- 
puter-implemented programming that correlates the 
response from an object illuminated by illumination light 
from the illumination light source with the known energy 
spectrum and therefrom determines the intrinsic object-color 
of the object. The systems can comprise further at least one 
light-blocking element surrounding the illumination light 
and the probe such that substantially no light other than the 
illumination light contacts the object at a location being 
detected by the probe. 

[0025] In further aspects the present invention provides 
reference data sets comprising an illumination -dependent 
color of at least one object under at least two different 
lighting conditions. The reference data set can comprise the 
illumination-dependent color of an object for at least two 
different types of lighting conditions selected from the group 
consisting of artificial lights, different latitudes, different 
altitudes, different times of day, different weather situations, 
and different altitudes. The object can be static or dynamic, 
depicted in a sequence of images that together depict move- 
ment or change. 

[0026] These and other aspects, features and embodiments 
are set forth within this application, including the following 
Detailed Description and attached drawings. The present 
invention comprises a variety of aspects, features and 
embodiments; such multiple aspects, features and embodi- 
ments can be combined and permuted in any desired manner. 
In addition, various references are set forth herein, including 
in the Cross-Reference To Related Application, that discuss 
certain apparatus, systems, methods or other information; all 
such references are incorporated herein by reference in their 
entirety and for all their teachings and disclosures, regard- 
less of where the references may appear in this application. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0027] FIG. 1 is a graph depicting the intrinsic wave- 
length-dependent response of an object. 

[0028] FIG. 2 is a graph depicting the reflectance spec- 
trum of the object from FIG. 1 under four different illtmii- 
nants or energy spectra. 

[0029] FIG. 3 is a graph depicting the relative distribution 
of X, Y, Z values for the object from FIG. 1 tmder four 
different flluminants. 

[0030] FIG, 4 is a graph depicting the relative distribution 
of red, green, blue digital image values for the object from 
FIG. 1 under the four illuminants from FIG. 3. 
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[0031] FIG. 5 is a depiction, presented both in color and 
numerically, of the digital RGB color of the object frona 
FIG. 1 under four different illuminations based on values 
from FIG. 4. 

[0032] FIG. 6 is a block diagram and flow chart exem- 
plary of a software hierarchy according to the present 
invention siiitable for implementing certain aspeas of the 
invention. 

[0033] FIG. 7 is a block diagram and flow chart exem- 
plary of an approach according to the present invention to 
determining the object color. 

[0034] FIG. 8 is a block diagram and flow chart exem- 
plary of an approach according to the present invention to 
analyzing an object color measurement and encoding the 
results in a color look up table (GLUT). 

[0035] FIG. 9 is a block diagram and flow chart exem- 
plary of an approach according to the present invention to 
associate the color of an object with the color of an object 
in an image. 

[0036] FIG. 10 is a block diagram and flow chart exem- 
plary of an approach according to the present invention to 
modify the color of an object in an image for desired 
illumination conditions and correcting it for the output 
device color space. 

[0037] FIG. U is a block diagram and flow chart exem- 
plary of an approach according to the present invention to 
implement a gamut-mapping algorithm. 

[0038] FIG. 12 is a schematic diagram exemplary of a 
color measurement system according to the present inven- 
tion incorporating a hand-held measurement probe. 

[0039] FIG. 13 is a block diagram exemplary of a color 
measurement module according to the present invention 
incorporating a light source, spectroradiometer and control 
interface. 

[0040] FIGS. 14 a-c are cross-sectional views of an 
example of a hand -held probe according to the present 
invention of a color measurement system. 

[0041] FIGS. 15 a-b are side front elevational views of the 
tip of the probe of FIGS. 14a-c. 

DETAILED DESCRIPTION 

[0042] The present invention provides devices, systems, 
methods and the like to control the apparent color of an item 
under difi^e^ent lighting or display conditions. The present 
invention can help provide accurate color for such items, 
both on the internet and in other situations, and can, if 
desired, determine whether a display screen is accurately 
displaying the color and also correct the image on the screen 
if it is not accurate. In some embodiments, the present 
invention comprises a probe suitable for taking accurate 
measurements of the intrinsic color characteristics, or intrin- 
sic wavelength-dependent response, of an object, and soft- 
ware or databases that provide information for a variety of 
lighting situations and light sources. The present invention 
also provides methods and apparatus for encoding desired 
color characteristics or information with an image such that 
the color of the image can be readily controlled at a remote 
display site, either at the display site itself or from another 
location. 



[0043] All terms used herein, including those specifically 
described below in this section, are used in accordance with 
their ordinary meanings unless the context or definition 
indicates otherwise. Also unless indicated otherwise, except 
within the clainjs, the use of "or" includes "and" and 
vice-versa. Non-limiting terms are not to be construed as 
limiting unless expressly stated (for example, "including" 
means "including without limitation" unless expressly stated 
otherwise). 

[0044] The scope of the present invention includes both 
means plus function and step plus function concepts. How- 
ever, the terms set forth in this application are not to be 
interpreted in the claims as indicating a "means plus func- 
tion" relationship unless the word "means" is specifically 
recited in a claim, and are to be interpreted in the claims as 
indicating a "means plus function" relationship where the 
word "means" is specifically recited in a claim. Similarly, 
the terms set forth in this application are not to be interpreted 
in method or process claims as indicating a "step plus 
function" relationship unless the word "step" is specificaUy 
recited in the claims, and are to be interpreted in the claims 
as indicating a "step plus function" relationship where the 
word "step" is specificaUy recited in a claim. 

[0045] A recorded image, of a scene or an object in a 
scene, is an array of information that represents the light 
emitted from the scene or object. The spectrum of such 
emitted light depends on the illumination provided by one or 
more light sotirces, and is collected by a lens or other optical 
device, and focused onto a photosensitive surface. When 
that array of information has been converted to an array of 
numbers it becomes a digital image. A digital image can be 
created by direcdy with a digital camera or indirectly using 
a digital scanner on a black and white or color photograph 
(or other medium). Digital cameras and scanners are known 
in the art and are commercially available from suppliers 
such as Kodak Company, Rochester, N.Y and Hewlett- 
Packard, Palo Alto, Calif. 

[0046] Digital images can be represented in a variety of 
ways. One common way 2-D approach is to define an array 
of points in an image using an X-Y rectangular coordinate 
system and to assign a color value at the position defined by 
the coordinates; similar approaches can be used for 3-D 
images as weU. Such a point in an image is known as a pixel. 
The numerical value of the pixel represents the color at that 
point in the image. 

[0047] The color response of the imaging device is usuaUy 
matched to the color response of the human eye. A common 
way of defining a pixel's color is to assign a numeric value 
to represent the intensity of each of the red, green and blue 
colors detected by the imaging device. This is referred to as 
RGB color and can, if desired, be defined by three 8-bit 
digital words, representing 256 possible intensity values for 
each of the R, G and B values. This color-encoding scheme 
allows definition of over 16 million colors. 

[0048] Most imaging devices are made color sensitive by 
positioning a wavelength selective optical filter in front of 
the image sensor. This filter selects light from either the red, 
green or blue regions of the visible light spectrum. The light 
detected by the imaging device will be the integrated inten- 
sity of the light illtmiinating the object, reduced by any 
aperture in the imaging device, the light absorbed by the 
object and the light absorbed by the red, green or blue color 
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filter (or other color(s), if desired) in front of the image 
sensor, as well as any other filters or optical elements in the 
image path. In a digital device the integrated intensity may 
be approximated by a sum of integrated intensities, each 
covering a small wavelength range and the sum substantially 
covering the wavelength range of visible light, for example: 

EcamaZjoo'*" Eill-Sjso^ Eref-S^eo'®" Efilt 

[0049] Other summations can also be used, and will be 
apparent to persons of ordinary skill in the art in view of the 
present application. 

[0050] The ability of an object to reflect or absorb light is 
an intrinsic characteristic of an object, and is not always easy 
to predict (although sometimes it can be) but can be mea- 
sured. These intrinsic characteristics are known as the intrin- 
sic-object color. When the object is a real item the charac- 
teristics can be referred to as the intrinsic wavelength- 
dependent response of the object. When the object is an 
artificially-created object, such as a computer generated 
cartoon or a model of an airplane in a CAD design work- 
station, the intrinsic-object color is made up of predeter- 
mined assigned colors. Intrinsic-object color can be artifi- 
cially re-assigned to a different value via manipulations by 
a user For example, the a web -site can show a given car 
model, then show it having different colors of paint. Using 
the features of the present invention, the car maker can show 
each of the different paint colors under a variety of different 
lighting conditions. There are many billions of possible 
visible light absorption or reflection, or other light emission, 
profiles for an object. 

[0051] Intrinsic-object color does not change when differ- 
ent light sources are shone on the object; intrinsic-object 
color is an inherent characteristic of the object. Rather, the 
illumination-dependent color of the object changes as the 
illumination of the object changes. In other words, an object 
may have a different perceived color under different lighting 
conditions, but the object itself does not change, only the 
perceived color changes. 

[0052] A "white light source" is an iUumination system 
that will allow the adapted human eye to perceive a white 
object as white. White light sources and other light sources 
commonly encountered in daily fife typically have known 
spectral characteristics, or energy spectra, and the energy 
spectra of the light soiu-ce can be determined empirically or 
otherwise as desired. 

[0053] If the specific absorption and reflectance, fluores- 
cence and other light emitting characteristics of an object are 
known, i.e., the intrinsic wavelength-dependent response or 
intrinsic-object color, then the perceived color of the object 
under various illumination conditions can be calculated in 
view of the present invention. This can be done by measur- 
ing the intrinsic wavelength-dependent response of an object 
and calculating the reflectance or other light emission spec- 
trum under a specific illumination, and then applying rela- 
tive color stimulus functions, e.g., a tristimulus function, to 
the emission spectrum to provide a quantitative measure of 
the observed energy spectrum, and then displaying that 
observed energy spectnm}. 

[0054] The response of ao imaging system such as a 
camera or the human eye is generally predictable and has a 
well-defined set of "normal" characteristics. The tristimulus 
response of the human eye is documented, for example, in 



a several versions of the commonly accepted published 
standard of the CIE known as the CIE Standard Colorimetric 
Observer. 

[0055] From the tristimulus values one can calculate the 
luminance and chrominance values in a variety of possible 
color spaces and using a variety of color appearance models. 
These models and color spaces are know in the art and are 
discussed in detail in reference texts such as Color Appear- 
ance Models by Mark D. Fairchild. One useful color space 
in the field of digital imaging is one that defines separate 
components for limiinance and chrominance. A well 
accepted color space is the CIELAB space: By applying an 
appropriate set of mathematical transformations one can 
calculate the L*, a* and b* values of this color space from 
the measured spectral response of an object. 

[0056] Typically, the luminance value comprises at least 
one element, L*, and corresponds to the perceived lightness 
(light-dark scale) of an illuminated object and the chromi- 
nance comprises at least two values, a* and b*, which can 
correspond, respectively, to the red-green and yellow-blue 
responses as described in the color opponent theory of 
vision; other luminance and chrominance values are possible 
and within the scope of the present invention. Typically the 
primary cause of changes in light-dark values under uniform 
illumination is topology. 

[0057] FIG. 1 shows an example of the intrinsic wave- 
length-dependent response of an object as measured by a 
spcctroradiometer. The object used for the Figure was color 
test pattern target number 6, bluish-green, from the Macbeth 
ColorChecker test pattern, a commonly used color standard. 

[0058] FIG. 2 shows the calculated reflectance spectrum 
for the object in FIG. 1 imder four different illumination 
conditions. The illumination values are iUuminants A, D50, 
D65 and Fll as defined by the QE. These illuminant 
examples are theoretical illumination values that correspond 
approximately to tungsten light (A), daylight (D50, D65), 
and fluorescent light (Fll). 

[0059] FIG. 3 shows the application of the CEE standard 
observer tristimulus functions to the calculated reflectance 
spectra from FIG. 2 to determine the tristimulus XYZ values 
for the object under each illuminant. 

[0060] FIG. 4 shows the digital RGB values calculated for 
a 24-bit RGB image pixel corresponding to the fiill bright 
color for the object. These values are determined from a 
mathematical transformation Cor converting from the device 
independent XYZ color space to the device dependent RGB 
color space. 

[0061] FIG. 5 depicts conceptually, both in color and 
numerically (which numbers correspond to the colors), the 
color differences for the same color patch under four iUu- 
minants as rendered for an image display monitor and then 
reproduced in hardcopy. The numbers in the color patches 
correspond to the digital RGB values for each image square. 
The actual colors in the printed copy of this document may 
not accurately depict the colors due to variations in the 
printer system used to create this document or other factors, 
but the figure is useful to demonstrate the relative differences 
of the perceived color of a single object under different light 
sources. FIGS. 3-5 are exemplary methods of presenting a 
numerical or graphical representation of an intrinsic object- 
color or color illumination-dependent color of an object; 
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[0062] If a digital image of an object is captured under a 
particular known illumination system and the luminance and 
chrominance values are calculated for the object from the 
RGB image values, then it is possible to convert specified 
object color values in the image to the true color values of 
the object under a hypothetical user-specified, or desired, 
illumination. This can be accomplished digitally by applying 
color look-up tables (CLUTs) for selected object chromi- 
nance values. Thus, the CLUTs provide one example of a 
reference data set comprising illumination-dependent color 
of the object, either directly as a set of such data or by 
providing data from which the illumination-dependent color 
can be readily determined, typically by providing an illu- 
mination light source reference data set comprising wave- 
length-dependent intensity characteristics of a desired illu- 
mination light soiircc sufficient to define the wavelength- 
dependent energy distribution of the desired illumination 
hght source, and then combining a tristimulus function and 
the intrinsic wavelength-dependent response of the object 
and the illumination light source reference data set to 
calculate al least one set of tristimulus values for the object 
when illuminated by the desired illumination Ught som-ce, 
and therefrom determining the illumination-dependent color 
of the object. By adjusting only the chrominance values, 
such as the a* and b* values, the relative luminance signal 
that is primarily due to topology can be preserved while the 
color response is corrected. Alternatively, if desired, the 
luminance values, such as the L* values, can also be altered. 

[0063] To minimize data requirements and speed up image 
processing, these look-up values can be limited to only 
selected critical colors. An example of this could be an 
outdoor scene in a sporting goods image where the color of 
a red backpack could be corrected but the color of the rock 
on which it is sitting or the background foliage could be 
ignored. The resultant reduction in data volume is particu- 
larly important for images transferred digitally over com- 
puter networks. This is referred to as segmenting the desired 
object from the remainder of the image, and can be specific 
to the object, or to any pixel in the image that contains the 
same color characteristics as the object prior to the adjusting, 
in which case such similar pixels can be adjusted to have 
chrominance values substantially the same as the adjusted 
chrominance values of the adjusted object. The desired 
object can be segmented from the entire remainder of the 
image, or from at least one second object in the image, or 
otherwise. 

[0064] Two types of reflectance are diffuse reflectance and 
specular reflectance. When light interacts with an object it is 
either scattered or absorbed or passes through an object. 
When light is absorbed its energy is deposited in the object. 
When light is scattered it changes direction. Light may be 
forward scattered or back scattered. This type of scattering 
is often wavelength dependent. The back-scattered light that 
is not absorbed by an object is usually the primary compo- 
nent determining an object's color. Sometimes the absorbing 
or scattering particles may be suspended in a minimally 
absorbing or minimally scattering medium such as water or 
some types of plastic. If the surface of such a material is very 
smooth or polished and is of a different refractive index, it 
may reflect light from the surface without interacting with 
the material to any extent. This type of reflection is called 
specular reflection and the angle of reflection is usually 
equal to the angle of incidence. Such specular reflections 
wiU typically be of a different color than the object and 



therefore can be ignored if desired by properly selected 
CLLTR during color correction. Alternatively, specular 
reflections could also be modified if desired for a particular 
effect. Diffuse reflection tends to comprise light that has 
interacted more with the object and is usually what is 
described as an object's color. In this way the object color 
can be corrected without modifying the apparent gloss or 
sheen of the object by erroneously correcting the "high- 
lights" generated by specular reflection. Desirable selection 
of CLUTs can comprise limiting the colors to be corrected 
to chrominance values characteristic of non-specular or 
diffuse reflection. Alternatively one can also, or instead, 
correct highlights such as specular reflection to match the 
type of iUumination. 

[0065] Another property of an object's interaction with 
light is fluorescence. Fluorescence occurs when an object 
absorbs light of one energy level and then emits light of a 
lower energy level. For example, in some fluorophores, blue 
light (which has a shorter wavelength and therefore a higher 
energy) is absorbed and red hght (which has a longer 
wavelength and therefore a lower energy) is emitted. In 
some materials such as fluorescent paint, the light emitted 
from an object can contain more fluorescence than reflec- 
tance. If the fluorescence characteristic ^ectrum of an 
object is known it can be used to further enhance the color 
rendition model for an object. 

[0066] A representative sample of chrominance values for 
a variety of lighting and environmental conditions can be 
calculated and recorded in a data array that can be associated 
with an image file. Intermediate lighting and environmental 
effects on chrominance values can be calculated by inter- 
polation of the representative values. For example, the 
expected color for an object in sunlight at noon on April 15 
can be predicted by interpolating between values for an 
object in sunlight at noon on April 1 and on May 1. 

[0067] In order that object color correction is applied only 
to the desired object, image segmentation procedures may 
be used to define and mask the area of an object to be 
corrected. Image segmentation and object recognition soft- 
ware for other applications are known in the art and can be 
appUed in view of the present application. Image segmen- 
tation and object recognition software is available from 
companies such as National Instnmients Corp. of Austin, 
Tex. 

[0068] The spectral absorption and diffuse reflectance 
characteristics of the object or objects in the image can be 
measured using a portable spcctroradiomcter or other suit- 
able measurement device connected to a portable computing 
device, such as a notebook computer or hand-held personal 
computer. Alternatively these characteristics can be 
extracted from commercial or other databases of measure- 
ments, or otherwise as desired. 

[0069] In certain embodiments the portable computing 
device comprises a portable slate type computer equ^ped 
with a color display and graphical user interface, which 
computer can further contain or be combined with a spec- 
troradiometer or other spectral measiu'ement device and a 
reference hght source. Various computer-implemented pro- 
gramming (e.g., software) components can be added to the 
portable computing device for a variety of measurements. 
The computer-implemented programming, as well as refer- 
ence data sets and other information discussed herein, can be 
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Stored in a computer-readable memory, which memory 
devices are well known in the art and can be, for example, 
a permanently contained memory or an easily removable 
device such as a floppy-disc or a CD- or DVD-based 
memory device. The computer-implemented programming 
can also be a file that can be transmitted from one computer 
to another over a suitable line or link, such as the internet, 
the world wide web, and a local area network, via modalities 
such as optical fibers, cell phones, and hand-held devices 
such as Palm -pilots''^, that can carry the computer signal. 

[0070] Such a file can include the computer-programming 
of the present invention that adjusts the apparent color of an 
object or an image comprising an object that has been 
adjusted according to the present invention. The file can also 
contain an object that will be adjusted according to the 
present invention at a remote terminal. Preferably, the file(s) 
containing such an object to be adjusted firther comprise 
computer-programming that adjusts the apparent color of the 
object. Further, the object can be associated with an iden- 
tifiable tag, such as a cookie, that assists the adjustment of 
the image at a remote location or terminal by an adjustment 
program located in a different or central computer. Such 
assistance can include turning on the method of adjusting by 
an external actuator, or invoking computer-implemented 
programming that interrogates the display mechanism, such 
as a computer display, CRT, or HDTV screen, determines the 
color characteristics of such display, then adjusts the object 
to provide a desired color to the object, preferably a color 
that accurately depicts the object under a desired illumina- 
tion light. Such interrogation programming is an additional 
feature of the present invention. Additionally, the identifi- 
able tag can be associated with a switch that can be tumed 
on or otherwise changed by the identifiable tag. 

[0071] A computer is a device, typically electronic, that 
can store coded data and can be set or programmed to 
perform mathematical or logical operations at high speed. 
The measurement devices are controlled by software and are 
interfaced to the computer system that is equipped with an 
operating system software such as Widows 2000 produced 
by Microsoft Corporation of Redmond, Wash. 

[0072] The software architecture of a preferred embodi- 
ment of the system comprises the system software, the 
spectroradiometer measurement software, illumination/ob- 
ject analysis software, and digital image color association 
software. 

[0073] Turning to certain embodiments of the data analy- 
sis and processing software and algorithms, illustrated in 
FIGS. 6-11, the computer software provides a graphical or 
text based (or a combination of graphics and text) user 
interface that provides a menu of commands and responds to 
a set of operator input instructions to control data analysis 
and processing. FIG. 6 depicts several components that 
measure an object's spectral absorption characteristics, or 
intrinsic wavelength-dependent response. FIGS. 7 and 8 
depict calculating the perceived color response of the object 
under various illumination conditions to give an illumina- 
tion-dependent color of the object. FIGS. 9, 10 and 11 depict 
associating the illumination-dependent color with a color 
value of desired pixels associated with the object in a digital 
image, thereby adjusting the color of the desired pixels using 
the illumination-dependent color to provide a different, 
desired color. 



[0074] In FIG. 6, the software contains measurement 
software that calibrates 602 the data acquisition hardware 
and controls the data acquisition or measurement of the 
object's color 604 and storage of the acquired data 610. It 
can further contain analysis software that can accesses 
various databases 606, 612, 616, analyze the measured data 
608, store the results 610, encode the results 614, and 
associate the results 618 with the image of the object 620. It 
can further contain image manipulation software that can 
access various databases 606, 612, 616, modify the image 
according to desired illumination conditions 622, interrogate 
the image-rendering device 624, correct the image for the 
desired image-rendering device 626 and finally render the 
image 628. 

[0075] As shown in both FIGS. 6 and 7, the measurement 
software can cahTjrate 602, 702 the wavelength response of 
the measurement module 604, which can be a spectroradi- 
ometric module, and apply the calibration correction factors 
thus derived to the data set collected from the measurement 
module 604. This software can further provide for calibra- 
tion of the intensity response per wavelength of the spec- 
troradiometric measurement module to a reference source of 
known intensity over a range of wavelengths and can apply 
the calibration correction factors thus derived to the data set 
collected from the measurement module 604. 

[0076] The measurement software can provide for acqui- 
sition/measurement 604, 704 and storage 610 of a dark or 
background spectrum signal data set and acquisition and 
storage of a measurement spectrum signal data set and 
subtraction of the dark or backgroimd spectrum signal data 
set from the measurement spectrum signal data set to 
generate a badcground corrected measurement spectrum 
data set. 

[0077] The measurement software can also analyze the 
measurement ^ectrum data set to determine if the measure- 
ment is of acceptable signal level, and can adjust the data 
acquisition integration time or number of samples integrated 
until acceptable signal level is obtained and provide an 
indicator to the operator if an acceptable signal level is not 
obtained within the range of adjustment of the measurement 
module. 

[0078] The measurement software then applies the wave- 
length and intensity calibration correction factors to the 
measurement spectrum data set, preferably a background 
corrected data set, to generate a calibrated measurement 
spectrum. The measured spectrum may be a reflectance 
spectrum or a fluorescence spectrum or a combined reflec- 
tance/fluorescence spectrum or other energy spectrum as 
desired. 

[0079] The measurement software can analyze the cali- 
brated measurement spectnmi data set to determine if the 
measurement is of acceptable quality and provide an indi- 
cator to the operator if the measurement quality is not 
acceptable. The measiu-ement software can then record or 
store 610 the acceptable calibrated measurement spectrum 
data set, measurement parameters, and performance indica- 
tors in a database such as an electronic or optical database. 

[0080] In FIG. 7, the measurement software contains or 
acquires a measurement spectrum data set, preferably cali- 
brated, from a reference object with known reflective prop- 
erties 706. The measurement software then acquires a mea- 
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suremeot spectrum data set, preferably calibrated, from an 
object that has been or will be imaged 708. The reference 
spectrum 706 and the object spectrum 708 are preferably 
corrected or adjusted in view of dark spectrum 704. The 
analysis software then calculates the object's intrinsic wave- 
length-dependent response from the two measurements 710. 

[0081] The spectral power P that a spectrometer measures 
at each wavelength, X, is a product of the spectral reflectance 
Rx of the object and the spectral power S;^^ of the illumination 
at that wavelength: 

[0082] Measuring the spectral power of an object P^ohi ^nd 
a reference surface P^ref using the same illumination S^^^, 
i.e., 

[0083] allows calculation of the object's intrinsic wave- 
length-dependent response, 

otj'^X tef^'x obi /^X tef 

[0084] In another embodiment the analysis software 
selects the object's intrinsic wavelength-dependent response 
714 from a database of previously measured or artificially- 
created wave length -dependent responses of the object 712. 

[0085] FIG. 7 illustrates how the analysis software can 
calculate object color 732 under a desired illumination by 
calculating 726 the interaction of the object's intrinsic 
object-color or intrinsic wavelength -dependent response 724 
with the normalized emission spectrum characteristics of the 
desired illiunination conditions 722. Such illumination con- 
ditions may be composed of illumination sources and envi- 
ronmental conditions and can include different artificial 
lights (such as incandescent, fluorescent, halogen and xenon 
lamps), latitudes, altitudes, times of day and weather situa- 
tions. 

[0086] The software can select the desired illiunination 
conditions 718 from a database of previously measured or 
artificially created iUumination conditions 716 or other 
source of spectral data, and/or the software can calculate 
desired illumination conditions algorithmicaUy 720 to pro- 
vide the desired iUumination 722. 

[0087] In another embodiment of the invention the soft- 
ware does not calculate the object color under the desired 
illumination 722 but selects it 730 from a database of 
previously measured or artificially measured object colors 
728 or other source of object colors. 

[0088] FIG. 8 illustrates an example of how the software 
can construct a GLUT 810 containing object colors under 
various illuminations by repeating the process of calculating 
the tristimulus values 806 from a selected wavelength- 
dependent response 802 and a selected flluminant 804 and 
transforming the tristimulus values to a luminance-chromi- 
nance space 808 for all desired iUuminants and aU desired 
colors for which intrinsic wavelength-dependent responses 
are available to the software. The Q.UT can be encoded 812 
and associated with the imaged object. 

[0089] The tristimulus values X, Y, and Z can be calcu- 
lated by applying color matching functions such as those of 
the CIE 1931 ColorimeUic Observer or the CIE 1964 
Supplementary Colorimetric Observer to spectral data that 



are directly measwed under a particular fllumination or 
created by calculating the interaction of an illuminant with 
an object's spectral absorption characteristics. Suitable for- 
mulae for calculating X, Y and Z values from spectral data 
using the color matching functions include: 



Y = k^4>{X)y{X)dX 



[0090] Fairchild disctisses the values of these variables in 
detail in color appearance models. The X, Y, Z values can be 
transforaied into coordinates for various other color spaces. 
In the CIELAB color space these values can be calculated 
using the equations. 

«^-5oo[oay^'^-(y/Kj'^] 
A*=2O0[(y/r^''^-(z/zj'^] 

[0091] As above, Fairchild discusses the values of the 
variables in detail in Color Appearance Models. The L*, a*, 
b* values can be calculated for the spectrum of each object 
of interest under eadi illuminant of interest and stored in a 
color look up table, or reference data set. Alternatively, in 
other embodiments of the invention they can be calculated 
as desired or required. The luminance-chrominance values, 
or the chrominance values alone, can then be used as 
described earlier to modify the chrominance values in an 
image to correct color. 

[0092] In another embodiment of the invention, as shown 
for example in FIG. 6, the analysis software records only the 
intrinsic wavelength dependent response of the object, and 
calculation of the luminance and chrominance values related 
to a particular illumination is performed by the image 
manipulation software 612, 616, 622. 

[0093] FIG. 9 ilhistrates an example of how object colors 
derived from measurements, or otherwise created, can be 
associated with the color of a depiction of the object in an 
image of the object. The analysis software can select an area 
in the image of the object that corresponds to the area where 
the object color was measured or for which it was calculated 
904. Such selection can be done with user interaction or 
automatically, e.g., by image segmentation. The analysis 
software can then convert the color values of the selected 
image pixels 906 from the image's color space 902 to 
luminance-chrominance space. The luminance and chromi- 
nance values can then be analyzed 908. A range of chromi- 
nance values can then be associated with the object cobr 
chrominance values 910. 

[0094] Image pixel color is often encoded as digital or 
analog RGB values. These values are used, for example, to 
create voltage signals to drive the electron guns of a CRT 
display or the TFTs or liquid crystals of a flat panel display, 
or otherwise to create a desired image on a suitable medium. 
Video signals encoded as RGB values typically assume a 
particular set of color phosphors. There are a limited variety 
of these color phosphors and they tend to be similar depcnd- 
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ing on what color standard was historically adopted or 
specified in that region. A new standard has evolved that has 
international agreement for High Definition Television 
(HDTV). This standard is being adopted for most contem- 
porary monitors used in studio video, computing, and com- 
puter graphics. This standard is referred to as Rec 709 in the 
color display indiistry and details can be found in SMPTE 
RP 177-1993 published by the Society of Motion Picture and 
Television Engineers. As one example, transformation of 
RGBCIEXYZ tristimulus values to an RGB color space is 
accomplished by performing a matrix multiphcation to 
produce 
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[0095] The inverse matrix that can be used to convert from 
RGB to CIEXYZ is 
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[0096] A variety of transforms allow color space conver- 
sion for images produced for one particular set of phosphors 
but being displayed on a different set of phosphors. See 
Television Engineering Handbook. 

[0097] In a preferred embodiment of the invention analy- 
sis of pixel values for this purpose can be accomplished by 
representing the range of dirominance values of the selected 
pixels in the form of histograms. If the CIELAB color space 
is used this would yield two histograms, one for the a* 
values and one for the b* values. Selection of a range of 
values from these histograms can be made by user selection 
or by another method, such as algorithmically by selecting 
the medians of the histogram values. 

[0098] Id a preferred embodiment of the invention the 
association of the selected ranges of image chrominance 
values with the chrominance values of the object color for 
one or more desired illuminations can be made by replacing 
the selected ranges of image chrominance values with the 
object chrominance values for one desired iUuminant and 
referencing the object luminance-chrominance values for 
other illuminations to this color. 

[0099] In another embodiment of the invention the soft- 
ware can create a GLUT that associates ranges of chromi- 
nance values of colors of the object in the image with object 
chrominance values for a range of illuminations. Other 
methods for making the association may also be employed. 

[0100] Once the association has been made the image data 
can be transformed back to the original image color space or 
to another color space of a desired image-rendering device 
such as a printer or a computer monitor. 

[0101] FIG. 10 illustrates an example of how the image 
depicting the object can be modified and corrected by the 
image manipulation software and algorithms using the 



image data and the associated object color data. The image 
can be modified for desired illumination conditions convert- 
ing the pixel colors 1004 of the image from the image cobr 
space 1002 to luminance-chrominance space and replacing 
the chrominance values of the object in the image 1008 with 
the object chrominance values for the desired illumination 
conditions 1006, 1012. The image can be further modified 
for the desired image-rendering device 1014 by converting 
the luminance-chrominance values of the modified image to 
the color space of the desired image-rendering device 1010. 
The image can also be corrected by mapping out-of-gamut 
luminance-chrominance values to values that the desired 
image-rendering device can render 1016, 

[0102] In a preferred embodiment the image manipulation 
software can use similar transformation algorithms as 
described above for the analysis software to convert the 
colors of the pixels in the image from the image color space 
1002 to a luminance-chrominance space 1004. In another 
embodiment the image manipulation software does not 
convert the whole image to a himinanoe-chrominance space 
but only those parts of the image to which the image 
modification process will be applied, which can be referred 
to as segmentation. 

[0103] The image manipulation software itself, or a user, 
can select a desired illumination condition for rendering the 
image 1006. The image manipulation software can then 
determine the chrominance values of the object's color for 
the desired illumination by extracting it firom an associated 
GLUT or database, or by various calculations or algorithms. 
In another embodiment the image manipulation software can 
calculate these chrominance values from a database of 
illumination data, such as those, 612, 622, depicted in FIG. 
6. 

[0104] The image manipulation software can then replace 
the chrominance values of the colors of the object in the 
image with the chrominance values of the object for the 
desired iUiunination 1008. 

[0105] In another embodiment the image manipulation 
software can determine the luminance values in addition to 
the chrominance values firom a GLUT or any of the methods 
described above 1012 for the chrominance values alone, and 
replace the luminance values in addition to the chrominance 
values in the image 1008. 

[0106] The image manipulation software can then convert 
the image &om the luminance chrominance space back to 
the original image color space or to the color space of the 
desired image-rendering device 1010, 

[0107] The image manipulation software can determine 
the characteristics of the image-rendering device by inter- 
rogating the device or another source of device character- 
istics such as a file or document 1014. Such device charac- 
teristics are known in the art and available in a variety of 
formats. A widely used example of such a device charac- 
teristic is an ICC profile as specified by the International 
Golor Consortium. The image manipulation software can 
then determine if the color values of any image pixels in the 
color space of the image rendering device are out-of-gamut 
and correct those values so that the device can render them 
1016. 

[0108] An example of such correction for an image that is 
to be displayed on an RGB computer monitor where each R, 



05/20/2004, EAST Version: 1,4.1 



us 2002/0012461 Al 



10 



Jan. 31, 2002 



G and B value has a maximum numerical value of 255 is a 
simple clipping procedure where, if the image modification 
and transformation resulted in any R or G or B values being 
greater than 255, those values arc replaced by 255. 

[0109] In a preferred embodiment of the invention the 
image manipulation software employs gamut-mapping algo- 
rithms to correct out-of-gamut color values. FIG. 11 illus- 
trates an example of a gamut-mapping algorithm where the 
image manipulation software first calculates a hull of the 
color space of the image-rendering device in luminance- 
chrominance space 1102. This hull can be calculated by 
converting all points on the outer surface of the device cobr 
space to luminance-chrominance space. Alternatively, a hull 
may be calculated only from selected points on the device 
color ^ace surface. Any out-of-gamut image pixels that 
have luminance-chrominance values that lie outside the hull 
1104 can be mapped onto the hull by calculating a desired 
point on the hull, typically the point closest to the out-of- 
gamut point 1106, and replacing 1108 the himinance- 
chrominance values of the out-of-gamut point with the 
luminance-chrominance values of this closest hull point. 
Alternatively, only the chrominance values may be replaced. 
After all desired out-of-gamut points have been mapped and 
the image has been converted to the color space of the image 
rendering device, the device will be able to render all the 
pixels of the corrected image. 

[0110] Turning to some depictions of systems and devices 
suitable for use in the present invention, FIG. 12 shows a 
schematic representation of a measurement system includ- 
ing a measiu'ement module 1201, which can comprise a 
spectroradiometer, can comprise an illumination unit and 
spectroradiometric measurement unit Spectroradiometric 
measurements are preferably made using a calibrated spec- 
trometer operated under software control by software 
installed on computer 1209 operably connected to measure- 
ment module 1201. Such spectrometers are known in the art 
and are commercially available firom suppliers such as 
Ocean Optics Inc. of Dunedin, Fla. 

[0111] A measurement probe 1203, typically hand-held, is 
coupled to the spectroradiometer and light source, for 
example by flexible fiber optic and control assembly 1205 
that directs the illumination light from the illumination 
source to the hand-held measurement probe and thence via 
the optical elements of the probe to the object 1207 to be 
measured. As one alternative, the spectroradiometer and/or 
the light source can be included in the band-held mecha- 
nism. The probe is placed in optical contact with the object, 
preferably via close proximity, physical contact or via a 
system such as an extensible sleeve that keeps undesirable 
light out of the measurement area. The diffusely reflected 
light from the object is collected by the optical elements of 
the hand-held probe and directed through the flexible fiber 
optic assembly to the spectroradiometer. 

[0112] FIG. 13 shows a block diagram exemplary of a 
preferred embodiment of a measurement module 1201. The 
measurement module comprises a light emitting diode 
(LED) light source 1301 comprising two or more LEDs 
1303,1305 and associated electronics. LED 1303 provides 
illumination suitable to determine certain fluorescence char- 
acteristics of the object to be measured. LED 1305 in 
combination with or separately from LED 1303 provides 
illumination suitable to determine certain reflectance char- 



acteristics of the object to be measured. Light from LED 
1305 and LED 1303 is directed via flexible light guide 1307 
through light guide assembly 1205 to hand-held probe 1203 
where the light is directed onto object 1207 to be measured. 
Flexible light guide 1307 may comprise one or more flexible 
light guides bundled together, or light may be optically 
directed from one or more light sources or light guides and 
mixed into one or more subsequent light guides which will 
direct the iUmnination light to the object to be measured. 
Measurement module 1201 further comprises spectroradi- 
ometer 1309 and switch and control interface 1311. light 
source 1301 and spectroradiometer 1309 are operably con- 
nected to interface 1311. Light source 1301, spectroradiom- 
eter 1309, and control interface 1311 are also operably 
connected via port 1313 to computer 1209. Spectroradiom- 
eter 1309 receives light reflected or otherwise emitted from 
object 1207 via light guide 1315. 

[0113] In some embodiments, the spectral measurement 
system, such as the spectroradiometer and associated com- 
ponents, can resolve the intrinsic wavelength-dependent 
response or iUumination-dependent response of an object, a 
light source, or other target to about 10 nm, typicaUy about 
5 nm, and preferably about 2nm. The system can comprise 
an iUumination light source having a known energy spec- 
trum and computer-implemented programming that corre- 
lates the response fi-om an object iUuminated by fllumination 
light from the illumination light source with the known 
energy spectrum and therefrom determines the intrinsic 
wavelength-dependent response of the object. 

[0114] Flexible light guide 1315 may comprise one or 
more flexible light guides bundled together, or light may be 
optically directed from one or more light guides and mixed 
into one or more subsequent light guides which will direct 
the illumination light to spectroradiometer 1309 to be mea- 
sured. Switch and control interface 1311 comprises elec- 
tronic components and may comprise microprocessors or 
micro-controllers that may control signals to and from LEDs 
1301, 1305, spectroradiometer 1309 and switches or other 
controls of probe 1203. Switch and control interface 1311 is 
operably connected to probe 1203 by control lines 1317. 
Control lines may transmit electrical or optical control 
signals and may be constructed firom one or more electrical 
or optical conductors. Light guides 1307 and 1315 and 
control lines 1315 are combined into flexible fiber optic and 
control assembly 1205 at port 1319. In a preferred embodi- 
ment of the invention flexible fiber optic and control assem- 
bly 1205 comprises a flexible tubular conduit that contains 
and protects the light guides and control lines between 
measurement module 1201 and hand-held probe 1203. 

[0115] The hand-held probe typically incorporates control 
switches to initiate a measurement, although the switches 
can be located elsewhere on the system if desired. For 
example, one control switch can be a two-position latching 
switch that determines whether the measurement wiU be 
triggered by a push-button or pressure sensitive switch. A 
second control switch can be a finger actuated momentary 
push-button switch that wiU trigger actuation if the push- 
button position is selected by the first switch. A third control 
switch can be a spring-loaded and/or pressure sensitive 
switch that triggers a measurement when the probe is placed 
in close proximity to the object 

[0116] In a preferred embodiment of the invention the 
probe tip can be shaped such to exclude undesirable light. 



05/20/2004, EAST Version: 1.4.1 



us 2002/0012461 Al 



11 



Jan. 31, 2002 



for example the probe can be formed such that the light- 
gathering element of the probe is recessed or is equipped 
with a flexible shroud that prevents ambient light from 
interfering with the measurement of the object. If it is 
desirable to have the point of measurement visible then the 
shroud can be removed and the measurement made in a 
room or chamber that excludes any ambient light that wotild 
confuse the measurement. 

[0117] FIGS. 14fl-c show several longitudinal cross-sec- 
tion views of a preferred embodiment of a hand-held probe. 
FIG. Ua and 14c are detail views of FIG. 14b. FIG. 14b 
shows the hand-held probe and a portion of the fiber optic 
and control assembly connected to it. The probe is shaped to 
be held like a pen in the hand of the operator. The probe 
comprises a substantially tubular component, or tube, 1401 
connected to tip assembly 1403 and end-plug 1406. End- 
plug 1405 comprises a substantially cylindrical object 
equipped with a concentric channel sized to accept and hold 
the outer conduit 1407 of the fiber optic and control assem- 
bly. End-plug 1405 is shaped to be fitted into tube 1401, 
Conduit 1407 passes through end-plug 1405 and most of 
tube 1401 and is inserted into a passage 1409 in tip 1403. 
Passage 1409 is shaped to allow conduit 1407 to be inserted 
a short fixed distance. Conduit 1407 may be secured in tip 
1403 and end-plug 1405 by a locking screw, adhesive, crimp 
or other suitable method of retention. 

[0118] Tip 1403 is substantially cylindrical with a tapering 
conical end 1411 and a concentric passage 1413. Tube 1415 
is fitted into passage 1413 and protrudes from conical tip 
1403. Flexible light guides 1417 are guided through tube 
1415 to terminate distally at measurement port 1419. The 
proximal ends of flexible Hght guides 1417 are directed back 
into conduit 1407 and thence to the illumination and spec- 
troradiometer assemblies. Tip 1403 also comprises a cham- 
ber that provides a platform 1423 on which may be mounted 
circuit board 1425 on which in turn switch 1427 may be 
mounted. Actuator button 1429 of switch 1427 is accessible 
to operator control through opening 1431 in tube 1401. The 
distal end of control wires 1433 are connected to switch 
1427 via circuit board 1425. Control wires 1433 pass into 
conduit 1407 and are connected proximally to a control 
interface board in the illumination and spectroradiometer 
assemblies. Switch 1427 comprises an input device that can 
provide operator input signals to the measurement and 
control software programs. 

[0119] FIGS. ISa-b are side front elevational views of a 
preferred embodiment of the invention, showing tapered 
conical end 1411 and a detail of tube 1415 and measurement 
port 1419 where flexible light guides 1417 comprise seven 
optical fibers arranged in a hexagonal array. 

[0120] The illimiination unit comprises a lamp and a lamp 
power supply. If desired, the unit can further comprise an 
optical relay system that accepts lUumination energy from 
the lamp and transmits it to the object. In a preferred 
embodiment of the invention the lamp is an LED-based 
lamp comprising one or more LEDs, for example from 
suppliers such as Nichia America Corporation. The lamp can 
also be a tungsten filament technical lamp with a known 
continuous temporally stable output spectrum, for example 
from Gilway Technical Lamp. 

[0121] The lamp can also be a pulsed lamp such as a 
pulsed xenon lamp. A pulsed lamp provides short duration. 



intense illumination, that when combined with high speed, 
and synchronized measurement data acquisition can signifi- 
cantly reduce the effect of ambient illumination on a mea- 
surement Pulsed xenon lamps are commercially available 
from suppliers such as Perkin-Elmer Corporation. 

[0122] The measurement unit comprises a spectrometer, 
preferably a spectroradiometer such as a calibrated spec- 
trometer, optically connected to the measurement probe, 
preferably a hand-held probe. The connection can be by 
flexible light guide comprising a fiber optic or liquid light 
guide. 

[0123] The illumination energy (e.g., the illumination 
Ught) is directed onto the object to be measured. This can be 
done in any desirable manner, for example via the flexible 
light guide and the directing optical elements of the hand- 
held probe, or directly by shining the light source onto the 
object. Light emanating from the object is collected by 
optical elements of the probe and is conducted to the 
spectrometer by a light guide such as a liquid hght guide, 
optical fiber, optical fiber bundle or a hollow fiber, that 
conducts light by intemal reflection from an entrance aper- 
ture to an exit aperture, or other suitable optical pathway. An 
optical fiber is a fiber composed of two concentric regions of 
optically transmissive material with different refractive indi- 
ces that transmits light axially by intemal reflection at the 
boundary of the two materials. In the case of gradient 
refractive index material this boundary may be diffuse rather 
than abrupt. 

[0124] In one embodiment, the spectrometer comprises an 
entrance port and optics to direct the beam to a wavelength 
dispersive grating or other spectrum-separating element and 
then to a CCD array detector or other suitable detector, 
typically a pixelated detector such as a CID detector, where 
the optical energy is transduced into electrical energy that 
can be measured. The array detector signal can be calibrated 
for wavelength and spectroradiometric response over a 
suitable wavelength range, for example in the UV, \^ible or 
NIR regions of the electromagnetic spectrum. Such spec- 
trometers are known in the art and are commercially avail- 
able from manufacturers such as Ocean Optics of Dunedin, 
Fla. Other wavelength dispersive elements including prisms, 
and other transducing detectors and detector arrays, are 
known in the art and can be employed in view of the present 
application. 

[0125] If desired, the spectrometer can be electrically 
connected to a serial interface board that performs analog to 
digital conversion of the measurement signal and then 
transfers this digital information via a data cable or other 
suitable connection to the serial data port of the system 
controller. Other architectures of analog to digital signal 
conversion and of digital data transfer to the system con- 
troller are known in the art and can be employed in view of 
the present application. 

[0126] From the foregoing, it will be appreciated that, 
although specific embodiments of the invention have been 
desaibed herein for purposes of illustration, various modi- 
fications may be made without deviating from the spirit and 
scope of the invention. Accordingly, the invention includes 
all permutations and combinations of the subject matter set 
forfli herein and is not limited except as by the appended 
claims. 
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What is claimed is: 

1. A computer-implemented method of adjusting at least 
one color of at least one object in a digital image comprising: 

a) providing the image containing the object and deter- 
mining the chrominance values of the object in the 
image; 

b) providing a reference data set comprising an illumina- 
tion-dependent color of the object that represents the 
color of the object under a desired illumination condi- 
tion; 

c) adjusting the chrominance values of the object in the 
image using the reference data set to provide a different 
desired color of the object in the image that represents 
the color of the object under the desired illumination 
condition, 

2. The method of claim 1 wherein the image includes at 
least a second object in the image. 

3. The method of claim 2 wherein the method further 
comprises segmenting the object from a remainder of the 
image. 

4. The method of claim 2 wherein the image comprises 
pixels and the method further comprises adjusting all pixels 
in the image that have chrominance values similar to the 
chrominance values of the object substantially the same as 
pixels corresponding to the object. 

5. The method of claim 1 wherein the method further 
comprises obtaining the illumination-dependent color by: 

a) providing an intrinsic-object color of the object to 
illumination; 

b) providing at least one illumination light source refer- 
ence data set comprising wavelength-dependent inten- 
sity characteristics of a desired illumination light 
source sufBcient to define the wavelength-dependent 
energy distribution of the desired illumination light 
source; 

c) combining a tristimulus function and the intrinsic 
object-color of the object and the illumination light 
source reference data set to calculate at least one set of 
tristimulus values for the object when illimiinated by 
the desired illumination light source, and therefrom 
determining the illumination-dependent color of the 
object 

6. The method of claim 1 wherein the method further 
comprises adjusting the image at a remote terminal of a 
computer network. 

7. The method of claim 6 wherein the computer network 
comprises an internet network. 

8. The method of claim 6 wherein the computer network 
comprises a fiber optic network. 

9. The method of claim 6 wherein the computer network 
comprises a wireless network as a carrier of the computer 
signal. 

10. The method of claim 6 wherein the method is imple- 
mented over the network such that the adjusting is per- 
formed at a remote terminal and computer-implemented 
programming able to perform the method is installed on the 
remote terminal in combination with the image. 

U. The method of claim 1 wherein the adjusting adjusts 
for different artificial lights. 



12. The method of claim 1 wherein the adjusting adjusts 
for at least one of the group consisting of different latitudes, 
different altitudes, different times of day and different 
weather situations. 

13. The method of claim 1 wherein the illumination- 
dependent color of the object is an artificially created 
illumination-dependent color. 

14. The method of claim 5 wherein the intrinsic-object 
color of the object is obtained using a numerical or graphical 
representation of the illumination-dependent color of the 
object in combination with a spectrum of a reference light 
source illuminating the image. 

15. The method of claim 1 wherein the method further 
comprises adjusting the illumination-dependent color of the 
object using gamut -mapping wherein the gamut is defined 
by image rendering capabilities of an image rendering 
device. 

16. The method of claim 1 wherein the image is associated 
with an identifiable tag that invokes the method of adjusting. 

17. The method of claim 1 wherein the image is associated 
with an identifiabb tag such that the adjusting can be 
initiated by an external actuator. 

18. The method of claim 16 wherein the identifiable tag is 
a file. 

19. The method of claim 16 wherein the adjusting is 
associated with a switch operably linked to be turned on by 
the identifiable tag. 

20. The method of claim 1 wherein the method is per- 
formed by a computer program that is combined with the 
image in a single file. 

21. The method of claim 20 wherein the object comprises 
a depiction of a real item and the intrinsic object-color of the 
real item is an intrinsic object-color of the real item. 

22. The method of claim 20 wherein the object comprises 
a depiction of an artificiatly-created object and the intrinsic 
object-color of the artificially-created object comprises pre- 
determined assigned chrominance values. 

23. The method of claim 1 wherein the method further 
comprises determining the luminance values of the object in 
the image. 

24. The method of claim 23 wherein the method further 
comprises adjusting the luminance values or using the 
luminance values to determine pixels to be adjusted. 

25. The method of claim 1 wherein the method further 
comprises limiting the colors to be corrected to chrominance 
values characteristic of diffuse reflection. 

26. A computer comprising computer-implemented pro- 
gramming able to perform the method of claim 1. 

27. A computer-readable memory comprising computer- 
implemented programming able to perform the method of 
claim 1. 

28. A computer-readable memory comprising an object 
potentially needing adjusting according to the method of 
claim I operably linked to an identifiable tag that invokes the 
adjusting from a remote location, 

29. A system comprising at least one probe able to detect 
an intrinsic object-color of an object to illumination and a 
computer comprising computer-implemented programming 
able to perform the method of claim 5. 

30. The system of claim 29 wherein the probe resolves the 
intrinsic object-color to about 5 nm. 

31. The system of claim 29 wherein the probe resolves the 
intrinsic object-color to about 2 nm. 
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32. The system of claim 29 wherein the probe comprises 
a spectroradiometer. 

33. The system of claim 29 wherein the system further 
comprises an illimiination light source having a known 
energy spectrum and the system comprises computer-imple- 
mented programming that correlates the response from an 
object illuminated by illumination light firom the illumina- 
tion light source with the known energy spectrum and 
therefrom determines the intrinsic object-color of the object. 

34. The system of claim 33 wherein the system further 
comprises at least one light-blocking element surrounding 
the illimiination light and the probe such that substantially 
no light other than the iUumination light contacts the object 
at a location being detected by the probe. 

35. A computer-contained reference data set comprising 
an illumination-dependent color of an object under at least 
two different lighting conditions. 



36. The reference data set of claim 35 wherein the 
reference data set comprises the illumination-dependent 
color of an object for at least three different types of lighting 
conditions selected from the group consisting of artificial 
tights, different latitudes, different altitudes, different times 
of day, different weather simations, and different altitudes. 

37. The reference data set of claim 36 wherein the object 
is an artificially-created object. 

38. The reference data set of claim 36 wherein the object 
is static. 

39. The reference data set of claim 36 wherein the object 
is depicted in a sequence of images that together depict 
movement or change. 

* * * f >K 
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